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TN THE CLAIMS 
The pending unamended claims axe reproduced below. 

1. (ORIGINAL) A method for solving, in a computer, stochastic control problems of 
linear systems in high dimensions, comprising: 

(a) modeling, in the computer, a structured Markov Decision Process (MDP), wherein a 
state space for the MDP is a polyhedron in a Euclidean space and one or more actions that are 
feasible in a state of the state space are linearly constrained with respect to the state; and 

(b) building, in the computer, one or more approximations from above and from below to 
a value function for the state using representations that facilitate the computation of 
approximately optimal actions at any given state by linear prograniming. 

2. (ORIGINAL) The method of claim 1, wherein the MDP comprises a supply chain 
planning process. 

3. (ORIGINAL) The method of claim 1, wherein the action space and the state space are 
continuous and related to each other through a system of linear constraints. 

4. (ORIGINAL) The method of claim 1, wherein the value function is convex and the 
method further comprises efficiently learning the value function in advance and representing the 
value function in a way that allows for real-time choice of actions based thereon. 

5. (ORIGINAL) The method of claim 1, wherein the linear function is approximated both 
from above and from below by piecewise linear and convex functions. 

6. (ORIGINAL) The method of claim 5, wherein the domains of linearity of the 
piecewise linear and convex functions are not stored explicitly, but rather are encoded through a 
linear programming formulation. 
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7. (ORIGINAL) The method of claim 5, wherein the domains of linearity of the 
piecewise linear and convex functions allow the functions to be optimized and updated in real- 
time. 

8. (ORIGINAL) The method of claim 1, wherein the value function can be efficiently 
approximated both from above and from below. 

9. (ORIGINAL) The method of claim 1, wherein the approximations can be repeatedly 
refined. 

10. (ORIGINAL) The method of claim 1, wherein the value function can be efficiently 
approximated from above based on knowledge of upper bounds on the function at each member 
of a selected set of states. 

1 1 . (ORIGINAL) The method of claim 1, wherein the value function can be efficiently 
approximated from below based on linear functions that lie below the convex value function. 

12. (ORIGINAL) The method of claim 1, wherein the value function can be 
approximated successively. 

13. (ORIGINAL) A computerized apparatus for solving stochastic control problems of 
linear systems in high dimensions, comprising: 

(a) a computer; 

(b) logic, performed by the computer, for modeling a structured Markov Decision Process 
(MDP), wherein a state space for the MDP is a polyhedron in a Euclidean space and one or more 
actions that are feasible in a state of the state space are linearly constrained with respect to the 
state; and 

(c) logic, performed by the computer, for building one or more approximations from 
above and from below to a value function for the state using representations that facilitate the 
computation of approximately optimal actions at any given state by linear programming. 
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14. (ORIGINAL) The apparatus of claim 13, wherein the MDP comprises a supply chain 
planning process. 

15. (ORIGINAL) The apparatus of claim 13, wherein the action space and the state space 
are continuous and related to each other through a system of linear constraints. 

16. (ORIGINAL) The apparatus of claim 13, wherein the value function is convex and 
the logic further comprises efficiently learning the value function in advance and representing 
the value function in a way that allows for real-time choice of actions based thereon. 

17. (ORIGINAL) The apparatus of claim 13, wherein the linear function is approximated 
both from above and from below by piecewise linear and convex functions. 

18. (ORIGINAL) The apparatus of claim 17, wherein the domains of linearity of the 
piecewise linear and convex functions are not stored explicitly, but rather are encoded through a 
linear programming formulation. 

19. (ORIGINAL) The apparatus of claim 17, wherein the domains of linearity of the 
piecewise linear and convex functions allow the functions to be optimized and updated in real- 
time. 

20. (ORIGINAL) The apparatus of claim 1 3, wherein the value function can be 
efficiently approximated both from above and from below. 

21. (ORIGINAL) The apparatus of claim 13, wherein the approximations can be 
repeatedly refined. 

22. (ORIGINAL) The apparatus of claim 13, wherein the value function can be 
efficiently approximated from above based on knowledge of upper bounds on the function at 
each member of a selected set of states. 



-4- 

G&C 3O879.79-US-01 



PAGE 7/22 * RCVD AT 11/23/2004 6:42:45 PM [Eastern Standard Time]* SVR:USPTO€FXRF-1/0 * DNIS:8729306 * CSID:+13106418798 * DURATION (mm-ss):06-44 



11-23^2004 03:55PM FROM-Gatss & Cooper LLP 



+13106418798 



T-659 P 008/022 F-822 



23. (ORIGINAL) The apparatus of claim 13, wherein the value function can be 
efficiently approximated from below based on linear functions that lie below the convex value 
function. 

24. (ORIGINAL) The apparatus of claim 13, wherein the value function can be 
approximated successively. 

25. (ORIGINAL) An article of manufacture embodying logic for solving stochastic 
control problems of linear systems in high dimensions, the logic comprising: 

(a) modeling a structured Markov Decision Process (MDP), wherein a state space for the 
MDP is a polyhedron in a Euclidean space and one or more actions that are feasible in a state of 
the state space are linearly constrained with respect to the state; and 

(b) building one or more approximations from above and from below to a value function 
for the state using representations that facilitate the computation of approximately optimal 
actions at any given state by linear prograinming. 

26. (ORIGINAL) The article of manufacture of claim 25, wherein the MDP comprises a 
supply chain planning process. 

27. (ORIGINAL) The article of manufacture of claim 25, wherein the action space and 
the state space are continuous and related to each other through a system of linear constraints. 

28. (ORIGINAL) The article of manufacture of claim 25, wherein the value function is 
convex and the logic further comprises efficiently learning the value function in advance and 
representing the value function in a way that allows for real-time choice of actions based thereon. 

29. (ORIGINAL) The article of manufacture of claim 25, wherein the linear function is 
approximated both from above and from below by piecewise linear and convex functions. 
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30. (ORIGINAL) The article of manufacture of claim 29, wherein the domains of 
linearity of the piecewise linear and convex functions are not stored explicitly, but rather are 
encoded through a linear programming formulation. 

31. (ORIGINAL) The article of manufacture of claim 29, wherein the domains of 
linearity of the piecewise linear and convex functions allow the functions to be optimized and 
. updated in real-time. 

32. (ORIGINAL) The article of manufacture of claim 25, wherein the value function can 
be efficiently approximated both from above and from below. 

33. (ORIGINAL) The article of manufacture of claim 25, wherein the approximations can 
be repeatedly refined. 

34. (ORIGINAL) The article of manufacture of claim 25, wherein the value function can 
be efficiently approximated from above based on knowledge of upper bounds on the function at 
each member of a selected set of states. 

35. (ORIGINAL) The article of manufacture of claim 25, wherein the value function can 
be efficiently approximated from below based on linear functions that lie below the convex value 
function. 

36. (ORIGINAL) The article of manufacture of claim 25, wherein the value function can 
be approximated successively. 
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